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decision makers and the planning needs of local educators. The 
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student attitudes, and other outcomes. Products of the system include 
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and reports geared toward the level of the school, class, and 
individual students. The system model also includes an outline of 
system capabilities and analysis and reporting requirements. 
Discussion protocols for articulating the information needs of model 
users are appended. (TJH) 
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EVALUATION FOR SCHOOL IMPROVEMENT: 
A MULTI-LEVEL, MULTI-PURPOSE MODEL 

This paper outlines a model for a comprehensive, multi- 
purpose, multi-user evaluation system designed to facilitate 
educational decisionmaking and to support school improvement and 
renewal. The model represents a unique "top-dowr bottom-up" 
approach which is school district-based but oriented to meet 
school building and classroom needs on the one hand and 
ultimately to serve state level needs on the other. In contrast 
to current evaluation practices which often feature overlapping 
and redundant evaluation requirements, the proposed model 
emphasizes the utility of common or compatible information bases 
to accomplish both the policy and accountability goals of 
district and state level decisionmakers and the planning needs of 
local educators, enabling them to address their unique problems 
and priorities. 

The conceptualization of the model draws on accumulated 
knowledge about what makes schools effective, about what makes 
evaluative information useful to teachers and administrators; 
about what makes information systems useful in organizations; and 
on the power of currently available, low cost microcomputer 
technology. 
Background 

The model starts with the assumption that evaluation 
information should be collected, analyzed, distributed and acted 
upon in order to improve educational quality, it assumes that 
such information can help upgrade educational quality by 



facilitating better educational decisionmaking, improved 
instructional planning and more effective school management at 
all levels of the educational hierarchy. District 
administrators or school principals, for example, could use valid 
information about student achievement, among other indicators, to 
make judgments about their schools' performance, to evaluate the 
effectiveness of particular programs, to establish grade, school, 
or district wide priorities, to allocate resources wisely, and to 
spot curricular or other problems needing correction. Using 
information about student test performance, attitudes, 
preferences, etc. in combination with their own perceptions, 
teachers might more easily and effectively accomplish such tasks 
as assigning students to groups, diagnosing individual learning 
problems, monitoring student progress, assessing subject matter 
mastery, and identifying students who need remediation or 
enrichment activities. A principal and teachers working together 
could use information about school context, instructional 
processes and outcomes to analyze local problems and improve the 
effectiveness their school programs. School board members and 
district leaders could likewise use such information to get a 
comprehensive and accurate picture of the quality of their 
schools and to target their improvement efforts accordingly. 

But while evaluation information has this potential, its 
power has yet to be realized. Why? The reasons are many and 
varied. Chief among these has been the source and nature of 
formal evaluation practice over the last two decades. Much of 
this practice has led to the proliferation of tests devoted to 
supplying the needs of legislators and administrators at the 



federal, state and later local levels who wished to know how 
mandated programs were working and how schools were achieving. 
These individuals with the power to distribute funds, to make 
laws, and to devise regulations were thought to be in the key 
influential positions to improve education and they have been 
quite influential in installing an overlapping, and to some 
extent redundant, set of evaluation requirements, e.g., national 
assessments, special program assessments, state assessments, 
local district assessments. The people at the bottom — teachers 
and local administrators — were sef.n as data providers rather 
than data users. They were seen as implementers of reform 
-ifforts rather than initiators of such efforts. Consequently, 
most of the attention in the evaluation community was directed 
towards providing information to assist ■top-down" 
decisionmaking, accomplishing at best only modest effects (Alkin 
et al, 1979; Cohen and Garet, 1975/ Patton, 1986). 

Teachers and local school administrators meanwhile questioned 
the validity of such evaluation efforts, saying that required 
tests did not reflect what they were teaching, that some were 
inappropriate for particular groups of students, and in any case, 
the results were often returned too late to be of any assistance* 
(Herman and Dorr-Bremrae, 1983). They claimed further that the 
paperwork and bureaucratic burdens associated with mandated 
evaluation requirements intruded into, rather than supported, 
their own planning and improvement efforts. They argued also that 
improvement of educational quality must must be directed at local 
school sites where teachers and administrators directly interact 



with children. "Bottom-up" needs, in short, were not and are not 
being well served by mandated evaluation and testing programs. 
Instead, teachers and principals developed their own sources of 
information, causing significant duplication of effort. 

Complementing these concerns were criticisms by some in the 
research community who questioned the value of these standardized 
test-dependent data bases for sound educational decisionmaking 
(Baker, 1983; Eisner, 1985; Sirotnik and Burstein, 1984). 
Criticized as providing a very limited view of educational 
quality, these tests, for the most part, examined student 
performance on only a narrow slice of the curriculum, emphasizing 
basic skills and giving little attention to learning in the 
content areas, higher-order reasoning skills, and the 
multiplicity of other academic, social, and vocational goals 
which schools are supposed to address. 

Not only did the test scores give an incomplete picture of 
the outputs of schooling, but they did so in a form that 
sometimes distorted their essential meaning. While the "How well 
are we doing" question provided impetus for much evaluation 
activity, answers framed solely in terms of test scores masked as 
much as they clarified, you simply couldn't backward chain from 
a simple test score to inferences about the overall quality of 
education in a state or district or at a particular school. The 
quality of school programs was (is) only one of many factors 
which contribute to student test scores. Cultural, social, 
economic, demographic and motivation factors are clearly 
influential, but often ignored in giving districts or schools 
report cards. Inequities and invalidities result, crediting 



schools which serve advantaged populations and disadvantaging 

schools serving minority and poor students. 

But even if credible testing instruments were available, more 

broadly-based tests were administered, and the results were to be 

integrated within a social/economic/community context, there 

would remain a further, serious deficiency in many previous 

evaluation conceptualizations. Evaluation in support of school 

improvement at the local level should not be limited to the type 

of data typically collected: outcome data. Left undocumented by 

evaluations focussing only on outcomes are the processes and 

contexts which create or contribute to those outcomes. 

Understanding these is critical to directing an effective agenda 

for school improvement. Eisner speaks to this point, 

"If we want to understand why we get what we get from our 
schools we need to pay attention not simply to the scores, 
but to the ways in which the game is played." (1985, p. 6) 

Concern with describing the educational process and its 

context, e.g., teaching practices, teacher morale, class climate, 

parent support, has been expressed by many in the profession 

(Goodlad, 1965, 1975a, 1975b; Sarason, 1982 on the culture of the 

school; Doyle, 1981 on classroom context; conceptualizations 

offered by Dreeban, 1967; Brof enbrenner , 1976; Tihunoff, 1979; 

McKenna, 1981; Sirotnik & al, 1983). But description and 

,> measurement of these elements often is ignored in routine 

evaluation practice. 

School context has been neglected not only as an source of 

explanatory hypotheses about why outcomes are as they are but 

also as an important intervening factor which influences how 

ER?C , p 



evaluation data themselves are interpreted and how they are used 
for school improvement and change (Sirotnik et al, 1985; Dorr- 
Breinme, 1984). Having technically sound, comprehensive data 
available does not assure that anyone will look at them, analyse 
them, discuss them, or take action stimulated by them. A growing 
literature on factors which influence evaluation utilization 
(Alkin et al, 1979, 1985; Bank and Williams, 1985), on factors 
which contribute to change and innovation in schools (Berman & 
McLaughlin, 1977; Sarason, 1982; Heckman et al, 1983) and on 
factors that affect the implementation of evaluation and 
information systems in fields outside of education provides clues 
on some of the socio-organizational-political issues involved in 
knowledge utilization — factors such as leadership support, 
ownership, perceived relevance, fit with routine practice, 
incentives, etc. which can be expected to influence whether 
evaluation information is acted upon and used +o alter existing 
practices. 

The above analysis suggests some of the reasons why 
evaluation has had only peripheral influence on teachers, 
principals or district personnel in their efforts to improve 
educational quality. To summarize: evaluation has been 
primarily linked with "top-down," highly centralized improvement 
approaches which were not necessarily sensitive to "bottom-up" 
needs; evaluation data was derived primarily from tests of 
student achievement which examined only a narrow range of 
outcomes; evaluations ignored critical variables in the context 
and process of schooling; evaluations did not sufficiently 
consider the factors which would ensure attention to findings and 
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transformation of findings into an action agenda. 

But there are possibilities for rethinking evaluation 

systems so that they serve multiple users and their diverse 

information needs. Many districts are currently moving in this 

direction (Williams and Bank, 1984; Williams and Bank, 1985; 

Idstein, 1985; Dussault, 1985). Radical changes in evaluation 

thinking are emerging which reflect both the reality of our 

decentralized or "loosely coupled" educational system and the 

awewome power of computers. Futurists such as Alvin Toffler 

predict an emergent "third wave" which will revolutionize the way 

our society and our educational institutions approach problems. 

Some problems cannot be solved on the local level. Others 
cannot be solved on the national level. Some require action 
at many levels simultaneously. Moreover, the appropriate 
place to solve a problem doesn't stay put. It changes over 
time. 

To cure today's decision logjam.... we need to divide up the 
decisions and reallocate them — sharing them more widely 
and switching the site of decisionmaking as the problems 
themselves require. (1980, p. 431) 

Education comes down to what happens to students in 
classrooms and in schools. American schools and classrooms 
encompass tremendous diversity in student population, in teacher 
skills, in curricular goals, in teaching strategies. Because of 
this diversity as well as because the actual control over 
instruction is in the school building, rather than in more remote 
and larger administrative units, the appropriate unit for solving 
many educational problems is the school (Goodlad, 1983; Baker, 
1983). Consequently, school personnel are among the appropriate 
beneficiaries of improvement-oriented evaluation systems. But 
individual schools may not have sufficient resources, expertise, 
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control, etc. to solve all their educational problems by 
themselves. The solution of many educational problems, as the 
Toffler (1980) quote implies, requires initiative, direction, 
resources and/or actions at higher administrative levels, levels 
which have legal responsibilities for governance, personnel 
resource allocation, and policy formation, among other things. 
These realities suggest the desirability of a distributed system 
of evaluation which would provide local shcools with a rich, 
locally sensitive information base for their problem-solving but 
which would also provide appropriate aggregate information for 
decisionmaking at higher levels of the system. Such a system 
echoes John Naisbitt's vision of the future, ■ ...institutions 
will be organized according to a management system based on the 
networking model. Systems will be designed to provide both 
lateral and horizontal, even multidirectional and overlapping 
linkages. 11 
The. Mfldfil 

Based on the foregoing analysis of problems in currenw 
evaluation practice, what would be a more productive model for 
improving the quality of schools? The answer flows directly from 
the analysis. An ideal system: 

1. makes relevant information easily available to teachers, 
school administrators, and district and state policymakers 
to aid their decisionmaking; 

2. enables efficient sharing of information within and 
across levels of the educational hierarchy, minimizing 
redundant, overlapping testing and evaluation requirements; 

3. includes information on a range of school outcomes; 

4. includes information on school context and student 
characteristics to contextualize outcome and effectiveness 
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analyses; 

5. includes information on school and instructional processes 
to elucidate and analyze local problems and accomplishments; 

6. links outcome information with instructional process and 
school context data to provide explanatory power for 
findings; 

7. includes externally fixed elements tc assure sensitivity 
to the information needs at the district and state levels and 
variable, locally selected elements and measures of interest 
to school professionals; 

8. encourages data collection, analysis, and use over time; 

9. builds on organizational and management strategies to 
facilitate system use including such things as: 

-locating responsibility for defining the system dually 
at the school and district levels 

-facilitating ownership and flexibility for local school 
uses 

-assuring leadership support at the district and school 
levels 

-attending to specific information and reporting needs 
of all groups 

-making the system user-friendly and easily accessible 



The model, in short, features the use of a comprehensive 
information base about student characteristics, school context, 
school and instructional process and a range of outcomes that can 
be analyzed, arrayed, and appropriately reported to facilitate 
decisionmaking at the classroom, school, district, and perhaps 
state levels and to satisfy reporting requirements for special 
programs, (Figure 1 displays an overview of the model syrtem.) 
Critical to the model is that its constituent elements are 
collaboratively defined and its implementation managed to promote 
use; further, to facilitate use where education actually occurs, 
it is school-based. 
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Implementation of the model depends on users 9 answers to a 
number of key questions, including what information should be 
included in the data base? What resources are available for 
acquir . he data? How shall it be analysed and reported to aid 
decisionmaking at different administrative levels? How can its 
usefulness and effectiveness be enhanced. The discussion of each 
of these questions below draws on an initial field trial of the 
model with five participating school districts. 
What Information Should Be Included in the Information System? 

The model stipulates a wide range of information, including 
student characteristics, school context, school process, 
instructional process, and a range of student and other outcomes. 
Within each of these categories an almost unlimited set of 
particulars exist. Which ones should be included? There are no 
hard and fast rules, but rather criteria against which local 
selections may be made: Choose for inclusion information about 
those things which are necessary for required reports and those 
things which are of greatest interest, concern and use to local 
users. Information necessary for required reports is the easier 
to identify, and typically includes such things as student 
attendance, ethnicity, sex, special program participation and 
basic skills test scores. 

Information of special interest and concern to various user 
groups is more difficult to uncover; and the process of 
identifying it requires direct input and significant interaction 
among and between those groups. Among the questions that these 
groups might be encouraged to consider are the following: 

- What questions do you have about the quality and operation 
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of y>ur school program? 

- What are the most important problems facing your school or 
district? 

- What important decisions are upcoming? 

- What are your school improvement priorities? 

With regard to information about school and instructional 
processes, both educators' working knowledge and the research 
literature on factors that contribute to educational 
effectiveness and instructional quality are a rich source of 
ideas. The effective schools literature, for example, suggests 
school and classroom process variables that are potentially of 
interest in analyzing the strengths and weaknesses of school 
programs and in constituting alternative indicators of school 
quality, variables sucn as performance expectations for students, 
time on task, school leadership, etc. about which information 
could be collected. 

The important point is that intended users explore the 
information that might best help them accomplish their 
respcnsibiJ ities, both from the standpoint of the problems they 
f^tceive in their environment, from the standpoint of what we 
know about whet "good" educational practice, and from the 
standpoint of information which can be validly and rather easily 
collected. Then, prioritize those information needs to define a 
customized information base. Table 1 lists a sample of the kinds 
of variables/ information slots that might be included in a 
comprehensive system. What are the limits of the information 
elements that can be included? Technology provides one set of 
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limits in terms of the size of the data base that can be 
accommodated; data collection and processing poses another set of 
limits in terms of the time and resources required for 
completion; and concerns for information overload on users poses 
additional limits. 

It is worth noting that the contents of the data base, once 
initially defined, need not — and best should not — stay 
constant over time. While some of the data about student 
characteristics and various performance indicators may in fact be 
relevant and required year after year, it is anticipated that 
various process and outcome variables would move on and off the 
data base, enabling schools and districts to investigate 
different areas within their environment over the years. Thus, 
for example, parent involvement might be perceived as a key 
problem initially, but after the problem is analyzed and acted 
upon, it ceases to be a concern, leaving room for the exploration 
of new areas. Or perhaps, like National Assessment, a district 
or school cycles through assessments in various subject areas in 
alternate years. Further, while some variables will be constant 
across the district, variable slots defined to meet the unique 
needs and interests of each school also will be necessary. 

Discussion guides to help potential users articulate their 
information needs and identify their priorities are included in 
Appendix A. 

How Can the Identified Data be Acquired? 

Much of the required data will already be resident within 
school district or school filer, e.g., information about student 
characteristic3 (sex, ethnicity, perhaps socioeconomic status); 
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Table 1 
Sample Data Base 



Background Information About Students 
Age 

Grade level 
Sex 

Ethnic background 

Time at current school 

Time 1n district 

Attendance/absence rate 

Sodc economic status 

Language status 

Special program participation 



Information on Student Outcomes 

Reading achievement 
Math achievement 
Attitude toward Reading 
Attitude toward Math 
Attitude toward School 



Classroom Processes 

Use of Instructional time 
Expectations of achievement 
Amount of homework 
Use of Individualized instruction 



School Context 

School climate 
Parent participation 



special program participation, standardized test scores, results 
of other district or school mandated tests, attendance records, 
time student has been resident in a particular district or in a 
particular school. Acquiring the data in these cases is a matter 
of rostering it by ID number, if necessary, and entering it by 
hand or electronically in the system, and/ or uploading or 
downloading it from other data bases (e.g., district or state) 
according to a specified structure. 

While the technological demands of the data access and entry 
should not be underestimated, acquiring data on the other 
information elements in the sy3tem also represents some 
challenge. This will require new data collection efforts, 
probably principally quef-.ionnaire-based because of resource 
constraints. Such data, however, could be augmented by entries of 
systematic teacher s f or others 1 observations/judgments/ 
perceptions. The source of the new data will be determined by 
the nature of the information desired, with the closer to the 
primary source the better, e.g., student, teacher, principal, 
parent questionnaires depending on the information need. Several 
compendiums exist to help schools with this task, e.g. 
questionaire items assessing a broad range of outcome and process 
issues for teachers, for students, for parents, and for 
administrators developed by CSE and a similar one including other 
data collection techniques compiled by Research for Better 
Schools. Computerized banking of these items, akin to similar 
ones for test item banks, would greatly facilitate the develop- 
ment process. Schemes for regularizing and systematizing 
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teachers 1 and other professionals 1 judgments and perceptions need 
further inquiry. 

Like the identification of information needs, users also could 
be profitably involved in the development, selection, 
modification and/or approval of these additional measures. Such 
involvement presumably would enhance the credibility of the 
results and thereby increase the probability of their use. 
flfla should ths, Data fie Analyzed and Reported? 

The analysis and reporting possibilities for a comprehensive 
data base are legion, particularly given users at the district, 
school, classroom levels representing a variety of different 
roles and responsibilities. Figures 2 to 4 display a sample of 
some of the possibilities at each level. 

Easily available computing power and sophisticated 
statistical software could enable almost any and all possible 
analyses to be conducted. The requirements of user-friendliness 
and of assuring that those who are novices in computers and in 
data analysis are able to access and use the information mean, 
however, that -he alternatives need to be made quite finite. 
Answers to several related questions are pertinent to identifying 
the most useful alternatives: What types of analyses should be 
conducted with the various data elements; at what level (s) and 
with what units should these analyses be conducted; and how 
should the results be arrayed and displayed? The answers to 
these questions, like those identified in previous sections, are 
user-based and need to be determined in close interaction with 
the users themselves. 

The types of analyses and the levels at which they are 




executed depend on the decisions to which the data is to 
contribute and the context of use. The process of selection, 
therefore, involves the interplay between what kinds of 
information is wanted and for what it is wanted; both questions 
offer potential starting points for helping users to articulate 
their needs, e.g., starting with what analyses are of interest 
and then moving to how this information could be used vs. 
starting with what decisions need to be made and then moving to 
what information and analyses would help make the decision. And 
moving back anri forth between the two may be necessary to help 
users deal with such abstractions. Among the types of analyses 
they will want to consider are the typical range: 

1. Descriptive analyses of individual elements of context, 

process or outcome data at a single point in time or over 
time. 

2. Outlier analyses identifying individuals, groups above or 
below certain critical values on specific indicators or 
above or below a critical range in the score distribution. 

3. Comparative or subgroup analyses which examine the 
performance of response of one special group and/or 
compares the performance of that group with another group, 
e.g., attitudes toward school of higher SES students versus 
those of lower SES; performance of students who have 
attended the same school for at least some minimum period 
of time; performance of Chapter 1 students in one district 
versus another. 

4. Relational analyses which examine relationships among and 
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between outcomes, instructional processes, and school 
context, e.g., is there a relationship between student 
attitudes and their achievement? To what extent is parent 
involvement related to student attitudes and their 
achievement? Has the parent education program influenced 
student achievement? 
The levels of analyses which can be provided also are 
straightforward: individual student; subgroup within class; 
class/teacher; school (for a particular grade or grade range); 
district; special program. Sample discussions guides are 
provided in Appendix B to help users identify their analyses and 
reporting needs. 

While these decisions may appear relatively simple and 
straightforward, each subsumes a number of difficult 
subquestions. The answers to some of these questions are driven 
by what are the technically appropriate analyses, but the answers 
to others should be grounded in what users find easiest to 
comprehend. For example, with regard to descriptive analyses of 
student outcomes, a number of possibilities are evident: means, 
medians, percentages falling and below a particular point; 
percent falling within a particular range. Student performance 
in reading, for example, might be characterized as mean 
percentile score, the percentage of student scoring above a 
particular percentile point; the percentage of students scoring 
within a particular percentile range, the mean performance of the 
top 25% of the class or school — or any of these indicators based 
instead on a stanine score, quartile score, NCE, or grade 
equivalent. while the devoted analyst might well want to 
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consider all these statistics, and ponder the situation from a 
variety of perspectives, the time, patience, and technical 
sophistication of teachers and school administrators may qrickly 
wear thin. Further, even given a particular statistic, the 
choice of meaningful cut-off or descriptive classification 
remains, e.g., if educators are interested in tracking over time 
their success in helping students achieve positive attitudes 
toward school, what score level represents a positive attitude, 
particularly if the assessment is not norm referenced? The range 
of data types likely to be resident within a data base, e.g., 
norm-referenced test data and criterion-referenced responses also 
brings with it additional difficulties in defining appropriate 
and standard benchmarks. For example, how do you define an 
outlier and does the definition or its computation change from 
one measure to the next? 

The choice of analyses goes hand in hand with display and 
reporting options. Appendix C illustrate a range of 
possibilities at the individual student, classroom, school and 
district levels. Interestingly, while most users in our 
exploratory study preferred the explanatory power of graphs and 
pictures, all respondent groups also wanted a roster on which 
they might capture information on as many variables as possible 
—so they might have all potentially relevant information "at a 
glance". Teachers wanted to know everything about individual 
students "at a glance;" principals wanted school level 
information and classroom summaries "at a glance"; and district 
personnel similarly wanted to quickly get a sense of the district 
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as a whole and individual school performance. The "everything at 
a glance," on a single page, concept probably reflects 
traditional reliance on paper records; while visually 
unappealling and lacking in power, they nonethless seem to 
provide a starting point for analysis. After noting interesting 
points of unusual strength or weakness, on these reports, users 
could call up a variety of graphs to investigate the situation in 
greater detail. For example, suppose a school district profile 
indicates that students' performance in reading and writing has 
remained at a relatively high level over the last two years, 
while mathematics performance appears to be slipping. District 
administrators might then call up graphical displays to see 
whether the slippage was district-wide or concentrated in 
particular schools, whether performance on all subscales of the 
test was similar or only certain ones evidenced particular 
problems, whether the performance problems seemed to be 
associated with particular groups of students, e.g., those who 
were new to the school, those with language difficulties, those 
with poor attitudes toward math, those who were frequently 
absent, etc. Additional data, if available, might then be used 
to explore in greater detail potential sources/causes and clues 
to their solution. 
Summary 

This report has offered a preliminary outline of a 
comprehensive information system that can help teachers and 
principals improve their schools and that can likewise serve the 
school improvement and policy agendas at the district and higher 
levels. It seeks to provide all those in the educational 
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enterprise with a fuller picture of the quality of existing 
programs and practices and with a tool for investigating 
strengths and weaknesses, exploring local concerns and diagnosing 
and responding to educational needs. 

A prototype data base management system for implementing such 
a system, based on D-base III Plus, is provided in the Appendix 
D. 
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Appendix A 



Discussion Guides for 
Articulating Information Needs 



Name 



District/School" 



Position 



Formulating Evaluation Questions 



The following questions are intended to help you think about your 
information needs and identify high priority evaluation questions. As you 
react to the questions below, you may want to consider Issues relevant to 
formulating district or school policy and/or those related to improving 
school practice . You'll probably find thd . some of the questions are more 
meaningful to you than others; concentrate on these. Please be as specific 
as possible 1n your answers. 

J? m *. What questions do you have about the quality of your education 
progran 



b. What kinds of information would help you answer these questions? 



2 a. What questions do you have about the operation of your school or 
district? 



b. What kinds of information would help you answer these questions? 



ER?C 



a. What are the most important problems facing your school or distri 



b. What kinds of information might help you solve these problems? 



a. What important decisions do you need to make about your 
school /district? 



b. What kinds of information would help your decision making? 
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5. What are your current evaluation mandates and/or reporting 
requirements? 



6. 



What are your school improvement priorities, e.g., have you targeted 
specific areas for action? 



Based on all of the above: WHAT ARE THE 5 MOST IMPORTANT QUESTIONS 
THAT THE EVALUATION SYSTEM SHOULD BE ABLE TO ANSWER. 



P9 
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Name 



District/School 
Position 

Student Outcomes 
and Characteristics 



Attached is a list of recommended student outcomes and characteristics 
to be included in the system. They were identified based on three 
considerations: feasibility of accurate measurement within project 
constraints, research evidence on indicators of school quality and 
effectiveness, and local school district experience with useful indicators 
for planning, reporting and decision-making purposes. 

Please consider the 11st attached and add any additional factors which 
you feel may be important to consider. Then mark the box next to each 
factor which you feel 1s definitely Important to consider. Note that 
starred Items * may require additional resources. Then rank the factors 
that you have marked as Important based on: 1) their importance as common 
factors 1n the system, and 2) their Importance as unique factors for your 
School. (Assign a "1" to the most Important factor, a "2" to the second 
most Important factor, and so on. Complete the ranking for eatn column 
separc **ly. ) 
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Student Outcomes 



Student Outcomes 
'» 

Standardized Achievement Test 
(norm-referenced) 

Reading 

Math 

Writing 

Other: 

District Competency Tests 
Reading 
Math . 
Writing 
Science 

Other • 

School or Classroom Curruculum Tests 
Reading 
Math 
Writing 
Science 

Other:^ 

' State Assessment 
Reading 
Math 
Writing 
Other: 



Definitely 
Important 



Attitudes Toward School 

Academic Self-concept 

Opinion of School Climate 

Attitudes Toward Other 
Racial /Ethnic Groups* 

Parent Satisfaction 
Other: 



Rankings 
Common Unique 
Factor Factor 
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B. Student Characteristics 

Student Characteristics 
Name 

Birthdate 
Sex . 
. 6r.>de 
Class 

Ethnic Background 

Language Status (FEP, LEP, NEP) 

Years at Current School 

Attendance/absences 

Program Designation 
(e.g*, Chapter 1) 

SES or Poverty Index 

Other; 



Definitely 
Important 



Rankings 
Common Unique 
Factor Factor 
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Name 



District/School 
Position 

Instructional Practices and Processes 
Context/Background Factors 

Attached is a list of potential Instructional process and 
context/background factors to be Included in the system. They were 
identified based on three considerations: feasibility of accurate 
measurement within project constraints, research evidence on Indicators of 
school quality and effectiveness, and local school district experience with 
useful Indicators for planning, reporting and decision-making purposes. 

Please consider the list attached and add any additional factors which 
you feel may be Important to consider. Then mark the box next to each 
factor which ycu feel 1s definitely Important to consider. Note that 
starred items * may require additional resources. Then rank the factors 
that you have marked as Important based on: 1) their importance as common 
factors 1n the system, and 2) their Importance as unique factors for your 
School. (Assign a "1" to the most Important factor, a "2" to the second 
most Important factor, and so on. Complete the ranking for each column 
separately. ) 



ERIC 
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A, Instructional Practi ces 



Instruction*! Practices 



Definitely 
Important 



Rankings 
Common Unique 
Factor Factor 



Primary Learning Goals/Objectives 

Homework Assignments 

Instructional Time 

Writing Assignments 

Adult/PupH Ratio 

Parent Volunteers 

Parent-teacher Contacts 

In-class Instructional Resources 
(texts, supplementary materials, 
equipment) 

Use of Out-of-class Instructional 
Resources (e.g., specialists, 
library, computer lab) 

In-class Computer Usage 

Uses of Testing/Assessment 

Multl -cultural Instruction 

Expectations for Conduct/Class 
Behavior 

Expectations for Achievement 
Other: 



ERIC 
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B. Context/Background Factors 



Rankings 

Definitely Common Unique 
Context/Background Factors Important Factor Factor 



Quality of Physical Environment 



Teacher Satisfaction 



Parent Involvement 



School Climate (suspensions, 
expulsions, vandalisms, safety) 

School-wide Instructional Resources 
(e.g., library, computer lab) 



Other: 



ERLC 
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Appendix 6 

Discussion Guides for 
Articulation Users' Analysis 
and Reporting Needs 



ERLC 
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GROUP CONSENSUS 
WHAT WE WANT TO KNOW 

GROUP 



This form asks you to think about the -types of information you want to receive from our 
project. The possibilities are almost limitless, and the mounds of data that could be 
generated could easily exceed your ability and desire to understand and use them. The 
usefulness of any of the data depends on you. What information do you really want? How will 
you use the information? The answers to these questions will help us provide reports that are 
relevant to your needs. 

The sections which follow ask you to think about three different types of information: 
descriptive, comparative, and relational. For each category of information, please specify the 
information or specific questions you want reported. Be sure you think about now you will 
actually use the information. Try to be as selective as possible. There are lots of 
interesting options; you will want to focus on those which will contribute to planning and 
decision-making. (In other words, "Wouldn't it be interesting...." is probably not sufficient 
justification.) A listing of available information is attached. 

Descriptive Information: Descriptive information simply provides student or teacher responses 
at either an individual level or summarized for the group. You will want descriptive 
information reported on all data that are interesting in their own right, e.g., student 
performance in reading, in math, attitudes toward reading, student preferences in materials. 
For each element, please indicate whether you want the information reported for each individual 
student summarized for each class, summarized for the school and/or summarized for the district 
as a whole. 



Level of Reporting 

What will it be Grade/ Grade/ 

Data Element used for? Indiv. Class School District 

12 3 4 

2 - 12 3 4 

3 - , 12 3 4 

4. 12 3 4 

5. 12 3 4 

6. 12 3 4 

7. , 12 3 4 

8. 12 3 4 

9. 12 3 4 

!0. 12 3 4 
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BSTnS or SS^TST ^* rative or 9™P ^stions. for our purposes, examine the 
SfS ^^r^rT«^.^'^^? : 2! P . and/0r ccmpares the Performance of that group 
readirperfoSce J hKS 5! ^ i*!"" 1 by any 5 taric ^«tle.) For example, how does the 
sSdents^ho Ze Jfn JI 9 5£ 5 'f^ 6 wlth those of lower SES students? How are 

have^b^ years parfonn^ compared to students who 

district compare with thoS i^rdulicS? ^formnce of Chapter 1 students in my 

Comparative or Group Questions of Interest mt will it be ^ for? 



1. 
2. 
3. 
4. 
5. 
6. 
7. 

8 « 
9 

10. 



ggMPM^ipns: Relational questions, for our purposes, looks at relationships among and 
between outcomes, instructional processes, and school context. For example, is there a 
^1^"^ ?t^f n i? ,dent and student *M««*nt? Is the amount of homework 
S^J^^lIS^ 7 eXtePt 15 * rent inVOlverent related to stu * nt 
Relational Questions of Interest What will it te used for? 

1. 



2. 
3. 
4. 

5 - . 

6 - . 
7. 

8 - . 
9. m 

10. 



ERIC ,S ' tosed 00 ttis review » are U* re elements that are of little interest? That is, 
asa^iformation that does not need to be collected? If please indicate: 



